Comparing generative and extractive approaches to information extraction from abstracts describing randomized clinical trials.

BACKGROUND: Systematic reviews of Randomized Controlled Trials (RCTs) are an important part of the evidence-based medicine paradigm. However, the creation of such systematic reviews by clinical experts is costly as well as time-consuming, and results can get quickly outdated after publication. Most RCTs are structured based on the Patient, Intervention, Comparison, Outcomes (PICO) framework and there exist many approaches which aim to extract PICO elements automatically. The automatic extraction of PICO information from RCTs has the potential to significantly speed up the creation process of systematic reviews and this way also benefit the field of evidence-based medicine. RESULTS: Previous work has addressed the extraction of PICO elements as the task of identifying relevant text spans or sentences, but without populating a structured representation of a trial. In contrast, in this work, we treat PICO elements as structured templates with slots to do justice to the complex nature of the information they represent. We present two different approaches to extract this structured information from the abstracts of RCTs. The first approach is an extractive approach based on our previous work that is extended to capture full document representations as well as by a clustering step to infer the number of instances of each template type. The second approach is a generative approach based on a seq2seq model that encodes the abstract describing the RCT and uses a decoder to infer a structured representation of a trial including its arms, treatments, endpoints and outcomes. Both approaches are evaluated with different base models on a manually annotated dataset consisting of RCT abstracts on an existing dataset comprising 211 annotated clinical trial abstracts for Type 2 Diabetes and Glaucoma. For both diseases, the extractive approach (with flan-t5-base) reached the best F 1 score, i.e. 0.547 ( ± 0.006 ) for type 2 diabetes and 0.636 ( ± 0.006 ) for glaucoma. Generally, the F 1 scores were higher for glaucoma than for type 2 diabetes and the standard deviation was higher for the generative approach. CONCLUSION: In our experiments, both approaches show promising performance extracting structured PICO information from RCTs, especially considering that most related work focuses on the far easier task of predicting less structured objects. In our experimental results, the extractive approach performs best in both cases, although the lead is greater for glaucoma than for type 2 diabetes. For future work, it remains to be investigated how the base model size affects the performance of both approaches in comparison. Although the extractive approach currently leaves more room for direct improvements, the generative approach might benefit from larger models.

Manual versus machine: How accurately does the Medical Text Indexer (MTI) classify different document types into disease areas?

The Medical Subject Headings (MeSH) thesaurus is a controlled vocabulary developed by the U.S. National Library of Medicine (NLM) for classifying journal articles. It is increasingly used by researchers studying medical innovation to classify text into disease areas and other categories. Although this process was once manual, human indexers are now assisted by algorithms that automate some of the indexing process. NLM has made one of their algorithms, the Medical Text Indexer (MTI), available to researchers. MTI can be used to easily assign MeSH descriptors to arbitrary text, including from document types other than publications. However, the reliability of extending MTI to other document types has not been studied directly. To assess this, we collected text from grants, patents, and drug indications, and compared MTI's classification to expert manual classification of the same documents. We examined MTI's recall (how often correct terms were identified) and found that MTI identified 78% of expert-classified MeSH descriptors for grants, 78% for patents, and 86% for drug indications. This high recall could be driven merely by excess suggestions (at an extreme, all diseases being assigned to a piece of text); therefore, we also examined precision (how often identified terms were correct) and found that most MTI outputs were also identified by expert manual classification: precision was 53% for grant text, 73% for patent text, and 64% for drug indications. Additionally, we found that recall and precision could be improved by (i) utilizing ranking scores provided by MTI, (ii) excluding long documents, and (iii) aggregating to higher MeSH categories. For simply detecting the presence of any disease, MTI showed > 94% recall and > 87% precision. Our overall assessment is that MTI is a potentially useful tool for researchers wishing to classify texts from a variety of sources into disease areas.

Affiliation Bias in Peer Review of Abstracts by a Large Language Model.

This study assesses affiliation bias in peer review of medical abstracts by a commonly used large language model.

Automatic Annotation of PubMed Articles with MeSH Qualifiers.

The Medical Subject Headings (MeSH) is a comprehensive indexing vocabulary used to label millions of books and articles on PubMed. The MeSH annotation of a document consists of one or more descriptors, the main headings, and of qualifiers, subheadings specific to a descriptor. Currently, there are more than 34 million documents on PubMed, which are manually tagged with MeSH terms. In this paper, we describe a machine-learning procedure that, given a document and its MeSH descriptors, predicts the respective qualifiers. In our experiment, we restricted the dataset to documents with the Heart Transplantation descriptor and we only used the PubMed abstracts. We trained binary classifiers to predict qualifiers of this descriptor using logistic regression with a tfidf vectorizer and a fine-tuned DistilBERT model. We carried out a small-scale evaluation of our models with the Mortality qualifier on a test set consisting of 30 articles (15 positives and 15 negatives). This test set was then manually re-annotated by a cardiac surgeon, expert in thoracic transplantation. On this re-annotated test set, we obtained macroaveraged F1 scores of 0.81 for the logistic regression model and of 0.85 for the DistilBERT model. Both scores are higher than the macroaveraged F1 score of 0.76 from the initial PubMed manual annotation. Our procedure would be easily extensible to all the MeSH descriptors with sufficient training data and, we believe, would enable human annotators to complete the indexing work more easily.Clinical Relevance-Selecting relevant articles is important for clinicians and researchers, but also often a challenge, especially in complex subspecialties such as heart transplantation. In this study, a machine-learning model outperformed PubMed's manual annotation, which is promising for improved quality in information retrieval.

Automated indexing using NLM's Medical Text Indexer (MTI) compared to human indexing in Medline: a pilot study.

Objective: In 2002, the National Library of Medicine (NLM) introduced semi-automated indexing of Medline using the Medical Text Indexer (MTI). In 2021, NLM announced that it would fully automate its indexing in Medline with an improved MTI by mid-2022. This pilot study examines indexing using a sample of records in Medline from 2000, and how an early, public version of MTI's outputs compares to records created by human indexers. Methods: This pilot study examines twenty Medline records from 2000, a year before the MTI was introduced as a MeSH term recommender. We identified twenty higher- and lower-impact biomedical journals based on Journal Impact Factor (JIF) and examined the indexing of papers by feeding their PubMed records into the Interactive MTI tool. Results: In the sample, we found key differences between automated and human-indexed Medline records: MTI assigned more terms and used them more accurately for citations in the higher JIF group, and MTI tended to rank the Male check tag more highly than the Female check tag and to omit Aged check tags. Sometimes MTI chose more specific terms than human indexers but was inconsistent in applying specificity principles. Conclusion: NLM's transition to fully automated indexing of the biomedical literature could introduce or perpetuate inconsistencies and biases in Medline. Librarians and searchers should assess changes to index terms, and their impact on PubMed's mapping features for a range of topics. Future research should evaluate automated indexing as it pertains to finding clinical information effectively, and in performing systematic searches.

How to Write a Comprehensive and Informative Research Abstract.

OBJECTIVES: This article provides practical guidance on developing a comprehensible abstract, including those required for funding applications, conferences, and publication. In addition, we discuss and demonstrate the practicalities of editing and revising an abstract for conference or peer review and identify emerging formats that may be more relevant to nurses and researchers. DATA SOURCES: This article has been informed by literature and the coauthors' respective experiences of preparing and reviewing abstracts for publication and conference presentation. CONCLUSION: Abstracts are a valuable tool to communicate the most important elements of the methods and results of a research project for a conference, manuscript, or even a research funding application. However, abstracts may often be an overlooked part of the dissemination process. An abstract determines whether or not a piece of research is relevant for presentation at a conference or valuable enough to be considered for peer review and subsequent publication. A strong and clearly written abstract positively predisposes reviewers of grant applications. IMPLICATIONS FOR NURSING PRACTICE: Writing an abstract is arguably the most challenging component of academic writing, summarizing the results of a substantive research project in three to five sentences and positioning them concisely within the background and implications for future practice, policy, and research. A well-written abstract is clear, concise, and critical and requires time and revision to ensure success.

Principales tecnologías de indización en las ciencias de la salud que se emplean en Occidente / Main indexing healthcare technologies used in West

En la actualidad, las tecnologías de indización en las ciencias de la salud están aportando mu-chos beneficios para el ámbito biomédico y la estandarización de su correspondiente termino-logía, puesto que esta cuestión es fundamental para lograr un diagnóstico médico más preciso e inequívoco Por esta razón, en este artículo se ha explicado con detalle cómo funcionan estas tecnologías: Terminología Anatómica In-ternacional (TAI), Medical Subject Headings y el Systematized Nomenclature of Medicine Cli-nical terminology (SNOMED CT), así como, las razones de la importancia de su uso para los sanitarios y los terminólogos.(AU)

Nowadays, healthcare indexing technologies are profiting the biomedical field and the stand-ardization of its corresponding terminology, since this is essential to achieve a more pre-cise and unequivocal medical diagnosis. Thus, in this article it has been performed a thorough explanation on how these healthcare technolo-gies work: International Anatomical Terminology (TAI), Medical Subject Headings and the Sys-tematised Nomenclature of Medicine Clinical terminology (SNOMED CT), as well as it was elucidated the reasons of its use for healthcare professionals and terminologists.(AU)

Licentîa hypomnema: reflections and perceptions in teacher education / Licentîa hypomnema: reflexões e percepções na formação de educadoras(es) / Licentîa hypomnema: reflexiones y percepciones en la formación del profesorado

Student protagonism is paramount in the knowledge construction process. In this paper, we discuss a didactic-pedagogical resource called licentîa hypomnema (LH), inspired in portfolios and learning diaries in which student-teachers record their understandings and reflections regarding pedagogical topics learned at University in a meta-learning process on learning about teaching. The initial context of the research was a Psychological Development and Teaching course, offered in the Teacher Education program at Universidade de Brasília (UnB). Two of the authors, then Literacy and Languages Teaching students, produced their LH and peer assessed each other during the course. In their annotations, the student-authors recorded two pedagogical situations related to the use of LH in their own teaching practices, causing an impact on their formative path. These materials consist of the data for analysis and discussion. Moreover, we discuss possible uses of reflexive writing in teacher education and other learning contexts. Producing a LH supports knowledge production and allows students to connect theory and practice, which consequently influences the student's teaching practice. Since LH is a didactic-pedagogical resource and not a close-ended tool, both the context and individuals who use it should be considered.(AU)

O protagonismo de estudantes tem grande importância no processo de construção do conhecimento. Neste artigo, discutimos um recurso didático-pedagógico chamado licentîa hypomnema (LH), inspirado em portfólios e diários de aprendizagem, isto é, estudantes de licenciatura escrevem suas compreensões e reflexões acerca de assuntos pedagógicos vistos na universidade em um processo de meta-aprendizagem sobre aprender a ensinar. O contexto inicial da pesquisa foi um curso de Desenvolvimento Psicológico e Ensino, ministrado em cursos de Licenciatura da Universidade de Brasília (UnB). Duas das autoras eram estudantes do curso de Letras, elas produziram seus LH, realizando avaliação por pares entre si durante o curso. As estudantes-autoras trouxeram em seus registros duas situações pedagógicas relacionadas ao uso desse recurso em suas próprias práticas de ensino que causaram um impacto em suas trajetórias formativas. São esses relatos que compõem os dados para análise e discussão. Além disso, procuramos discutir possíveis desdobramentos para o uso do recurso da escrita reflexiva na formação docente e em outros contextos de aprendizagem. A produção do LH fornece suporte para a produção de conhecimento e permite ao aluno conectar teoria e prática, o que, consequentemente, reverbera na prática de ensino do aluno. O LH é um recurso didático-pedagógico e não uma ferramenta fechada, portanto, tanto o contexto quanto as pessoas que o utilizam devem ser considerados.(AU)

El protagonismo de los estudiantes tiene gran importancia en el proceso de construcción del conocimiento. En este trabajo, discutimos un recurso didáctico-pedagógico llamado licentîa hypomnema (LH), que se inspira en diarios de aprendizaje, es decir, estudiantes de profesorado escriben sus comprensiones y reflexiones sobre temas pedagógicos aprendidos en la universidad, produciendo un proceso de meta-aprendizaje sobre aprender a enseñar. El contexto inicial de la investigación fue un curso de Desarrollo Psicológico y Enseñanza, impartido en el programa de Formación de Profesores de la Universidade de Brasília (UnB). Dos de las autoras eran alumnas de la carrera de Letras que produjeron sus LH y realizaron una evaluación por pares mutua durante el curso. Las estudiantes autoras plantearon en su LH dos situaciones pedagógicas relacionadas con el uso de este recurso en sus propias prácticas de enseñanza, causando un impacto en su trayectoria formativa. Estos materiales constituyen los datos para el análisis y la discusión en esta investigación. Buscamos discutir posibles desdoblamientos para el uso del recurso de la escritura reflexiva en la formación docente y en otros contextos de aprendizaje. La producción de este recurso da soporte a la producción de conocimiento y permite al estudiante conectar teoría y práctica, lo que consecuentemente repercute en su práctica docente. El LH es un recurso didáctico-pedagógico y no una herramienta cerrada, por lo tanto, se debe considerar tanto el contexto como las personas que lo utilizan.(AU)

Reporting characteristics of systematic review abstracts published in the proceedings of the SBPqO meeting

Aim: This study aimed to assess the reporting characteristics of systematic review abstracts published in the proceedings of the Sociedade Brasileira de Pesquisa Odontológica (SBPqO) meeting. Methods: We selected abstracts published in the SBPqO meeting proceedings of 2019 and 2020, mentioning that a systematic review was conducted in the title, objective or methods sections. One researcher performed the screening and the data extraction after a pilot test training. The following data were extracted: affiliation of the primary author, dental specialization, the term "systematic review" mentioned in the title, reporting of the objective, reporting of eligibility criteria, reporting of information sources, reporting of the number of included studies and if a meta-analysis was performed. A descriptive analysis of the data was performed with data summarized as frequencies. Results: We included 235 abstracts. A total of 20 studies were from the Universidade de Uberlândia (8.5%), and the main specialization was Restorative and Esthetic Dentistry, with 47 studies (20%). Most of the studies mentioned the term "systematic review" in the title (n=219; 93.2%) and reported the objective (n=231; 98.3%). A great majority of studies did not report the eligibility criteria (n=97; 41.3%) or it was classified as unclear (n=96; 40.8%). The great majority of studies only reported the databases searched (n=103; 43.8%) or databases and date of search (n=74; 31.5%). Most of the studies reported the number of included studies (n=204; 86.8%). Conclusion: Based on this study, the reporting characteristics of systematic review abstracts published in the proceedings of the SBPqO meeting are satisfactory. However, there is room for improvement

THE STRUCTURE FORMAT OF ABSTRACTS: A SURVEY OF LEADING DENTAL JOURNALS AND THEIR EDITORS.

OBJECTIVES: To present the actual usage of different structure formats in abstracts of randomized controlled trials (RCTs) and systematic reviews (SRs) published in SCIE-indexed dental journals, and to assess the awareness, knowledge, as well as attitudes towards the structured formats of RCT and SR abstracts among editors-in-chief (EICs) of dental journals. METHODS: In the first part of this study, we selected SCIE-indexed dental journals and assessed their eligibility according to pre-determined criteria. All RCTs and SRs published in the included journals during January-June 2020 were identified through a hand-search. The actual usage of different structure formats and headings, as well as relevant editorial policies were extracted. In the second part, an anonymous online survey among the EICs of included dental journals was conducted. RESULTS: A total of 88 journals were included, from which 364 RCT abstracts and 130 SR abstracts were identified. For RCT abstracts, 86% were structured, with 83% in IMRaD format (Introduction, Methods, Results, and Discussion) and 3% in highly structured (HS) format. For SR abstracts, 80% were structured, including 73% in IMRaD and 7% in HS format. According to the "instructions to authors", most journals required either IMRaD (68%) or HS (5%) for RCTs, while less than half required either IMRaD (36%) or HS (9%) for SRs. Twenty-one (24%) EICs participated in our survey, among which 18 agreed that structured formats could improve the reporting quality of RCT abstracts, while only 12 of them thought HS format should be widely recommended in the dental field. CONCLUSIONS: Compared with the HS format, IMRaD was more frequently used and required among RCT and SR abstracts in dentistry. Structured formats held a relatively high degree of recognition among EICs of dental journals. Joint efforts are needed for improving the awareness and usage of HS format.

Scientific X-ray: Scanning and quantifying the idea evolution of scientific publications.

The rapid development of modern science nowadays makes it rather challenging to pick out valuable ideas from massive scientific literature. Existing widely-adopted citation-based metrics are not adequate for measuring how well the idea presented by a single publication is developed and whether it is worth following. Here, inspired by traditional X-ray imaging, which returns internal structure imaging of real objects along with corresponding structure analysis, we propose Scientific X-ray, a framework that quantifies the development degree and development potential for any scientific idea through an assembly of 'X-ray' scanning, visualization and parsing operated on the citation network associated with a target publication. We pick all 71,431 scientific articles of citation counts over 1,000 as high-impact target publications among totally 204,664,199 publications that cover 16 disciplines spanning from 1800 to 2021. Our proposed Scientific X-ray reproduces how an idea evolves from the very original target publication all the way to the up to date status via an extracted 'idea tree' that attempts to preserve the most representative idea flow structure underneath each citation network. Interestingly, we observe that while the citation counts of publications may increase unlimitedly, the maximum valid idea inheritance of those target publications, i.e., the valid depth of the idea tree, cannot exceed a limit of six hops, and the idea evolution structure of any arbitrary publication unexceptionally falls into six fixed patterns. Combined with a development potential index that we further design based on the extracted idea tree, Scientific X-ray can vividly tell how further a given idea presented by a given publication can still go from any well-established starting point. Scientific X-ray successfully identifies 40 out of 49 topics of Nobel prize as high-potential topics by their prize-winning papers in an average of nine years before the prizes are released. Various trials on articles of diverse topics also confirm the power of Scientific X-ray in digging out influential/promising ideas. Scientific X-ray is user-friendly to researchers with any level of expertise, thus providing important basis for grasping research trends, helping scientific policy-making and even promoting social development.